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CN ' Abstract 

j^ , We show that for all 1 < A; < logn the fc-ary unbiased black-box complexity of the n- 

dimensional OneMax function class is 0(n/k). This indicates that the power of higher arity 
operators is much stronger than what the previous 0(n/logfc) bound by Doerr et al. (Faster 
On ■ black-box algorithms through higher arity operators, Proc. of FOGA 2011, pp. 163-172, ACM, 

2011) suggests. 

The key to this result is an encoding strategy, which might be of independent interest. 
[t1 I We show that, using fc-ary unbiased variation operators only, we may simulate an unrestricted 

^' ■ memory of size 0(2*^) bits. 

1 Introduction 

Black-box complexity theory tries to give a theory-driven answer to the question how difficult 
a problem is to be solved by general purpose optimization approaches ("black-box algorithms"). 
The recently introduced notion of unbiased black-box complexity in addition allows a distinction 
regarding the arity of the variation operators employed (see the theory track best-paper award 
winner by Lehre and Witt [LWIO]). The only result so far indicating that there exists a hierarchy 
^3 ■ of tbe unbiased black-box models with respect to their arity (that is, the only result indicating 

^N I that for any A; € N the {k + l)-ary operators are strictly more powerful than fc-ary ones) is the 

result of Doerr, Johannsen, Kotzing, Lehre, Wagner, and Winzen showing that the fc-ary unbiased 
black-box complexity of the n-dimensional OneMax„ function class is 0(n/ log fc), 1 < fc < n. 
^ i This function class contains for each length-n bit string z the function Om^(-) that assigns to each 

bit string x the number OMz{x) := \{j € [n] \ Xj = Zj}\ of positions in which x and z agree. In 
this work, we given an indication that the hierarchy of complexity models might be even much 
more pronounced than what [DJK"'"11] suggests. In fact, we show that for 1 < fc < logn the fc-ary 
unbiased black-box complexity is 0{n/k). In particular, the (irrespective of the arity) best possible 
complexity of 0(n/logn) is attained already for arity logn instead of a linear arity. 

We note that all other results on higher arity black-box complexities [DKLWll,DW12a,RVll] 
consider either only the binary or 3-ary black-box models or they consider the *-ary model, in which 
the arity of the operators may be arbitrarily large. As shown by Rowe and Vose in [RVll], for 
the latter model, the unbiased black-box complexity coincides with the unrestricted one by Droste, 
Jansen, and Wegener [D JW06] . 
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1.1 Black-Box Complexity 

Black-box complexity as a measure for problem difficulty was introduced by Droste, Jansen, and 
Wegener in their seminal paper [DJW06]. Roughly speaking, the black-box complexity is the 
least number of fitness evaluations needed to solve a problem. It thus is a lower bound for the 
performance of any evolutionary algorithm and any other randomized search heuristic. 

For the OneMax.„ function class, Droste, Jansen, and Wegener proved a lower bound of 
il(n/logn) via information theoretic arguments. That this bound is asymptotically tight was later 
shown by Anil and Wiegand [AW09]. As noted in [DW12b], both bounds were in fact known much 
earlier, in completely different fields of theoretical computer science (coin weighting games [ER63], 
Mastermind [Chv83]). 

Droste, Jansen, and Wegener also note that their unrestricted version of black-box complexity 
sometimes gives unexpectedly low complexities. For example, the NP-complete maximum clique 
problem has a black-box complexity of only 0{n'^) as witnessed by the algorithm querying all 
possible two-cliques, learning all edges by this, and from this information learning the maximum 
clique. As possible solution they suggest adding a memory restriction. As far as we know, this line 
of research, however, has not created much follow-up work — with one exception being the recent 
result by Doerr and Winzen that even restricting the memory to a size of one does not change the 
black-box complexity of OneMax„, cf. [DW12b]. 

A different way of restricting the class of regarded black-box algorithms was suggested by Lehre 
and Witt [LWIO]. Noting that many randomized search heuristics treat the search space in a 
symmetric manner, they suggest an unbiased black-box model in which (i) new search points can 
only be generated from previously queried ones (or may be sampled randomly from a uniform 
distribution) and (ii) all variation operators used for this purpose have to be unbiased, that is, 
treat both bit positions and bit values in a symmetric fashion. In addition to hopefully giving 
more meaningful black-box complexities, this model also allows a natural definition of the arity of 
an algorithm: an algorithm is called fc-ary if only variation operators are used that take up to k 
previous search points as input. So with this model, we also have a complexity theoretic tool to 
discuss the role of, say, crossover in evolutionary computation. 

In [DKLWll] and [RVll] different generalizations of the unbiased black-box model can be 
found. The model by Lehre and Witt works only for pseudo-Boolean functions, whereas Rowe and 
Vose [RVll] present a framework for general function classes and Doerr et al. [DKLWll] consider 
different unbiasedness notions for the single-source shortest paths problem. 

While in [LWIO] only 1-ary ("unary") variation operators were regarded, the work [DJK+ll] 
gives the first result indicating an increasing strength of higher arity operators. For the n- 
dimensional OneMax„ function class, they show an upper bound for the A:-ary unbiased black-box 
complexity of 0{n/logk), 2 < k < n. This bound is sharp for k = n '"•*. Together with the 
G(nlogn) bound for the unary case [LWIO], this proves an advantage of using higher arity oper- 
ators. While no lower bounds (apart from the cases k = 1 and k = n^^"^) are known, the upper 
bound of 0(n/ log A;) does not indicate huge gains from using variation operators of arity higher 
than 2. 

1.2 Our Result 

In this work, we show that the increasing power of higher arities may be much more pronounced 
than what seems to have been known so far. We prove that, for all 2 < A; < logn, the fc-ary unbiased 



black-box complexity is at most 0{n/k), replacing the previous logarithmic gain by a linear one. 
The proof of this result uses a number of arguments that might be of interest beyond the mere 
result. In particular, we design {k + 2)-ary operators that allow to simulate a memory of 2^^ bits in 
an unrestricted fashion (encoding technique). Also, to save memory space, we apply a derandomized 
version of the random sampling strategies used in the three papers proving the 0(n/ log n) upper 
bound for the unrestricted black-box complexity of OneMax„. This derandomized result says that 
for any n G N there exists a fixed sequence of 0(n/ log n) search points such that querying their 
fitness with respect to any ONEMAX„-function surely reveals the optimum of this function. The 
result itself was proven in [ER63]. However, our result seems to be the first application of the 
derandomized version in black-box optimization. 

We do give in that we do not have any new lower bounds for the /c-ary black-box complexity 
of OneMax„. In particular, we cannot rule out that already for k = 2 the unbiased black-box 
complexity is 0(n/logn). Unfortunately, currently no such bounds are known. This remains a 
significant open problem for which the current lower bound methods all seem to fail. 

2 The Unbiased Black-Box Model 

As mentioned in the introduction, some shortcomings of the unrestricted black-box model inspired 
Lehre and Witt [LWIO] to introduce a new, more restrictive black-box model. This is the unbiased 
black-box model. It is based on the observation that many randomized search heuristics employ 
only such variation operators that are unbiased. In this section, we briefly introduce the unbiased 
black-box model and we define unbiased black-box complexity. 

Roughly speaking, an unbiased variation operator must treat the bit positions l,...,n and 
the bit entries and 1 in an unbiased ("fair") way. In particular, an unbiased operator may not 
require a specific bit value to be set to or to 1. As mentioned, the unbiased model, in addition 
to excluding some highly artificial algorithms, also admits a notion of arity. A /c-ary unbiased 
black-box algorithm is one that employs only such variation operators that take up to k arguments. 

For a formal definition of the unbiased black-box model let us briefly fix some notation. For all 
positive integers fc S N we abbreviate [k] := {1, . . . , k} and [0..A;] := [k] U {0}. By e^ we denote the 
k-th. unit vector (0, . . . , 0, 1, 0, . . . , 0) of length n. The bitwise exclusive-or is denoted by ©. For 
n G N, by Sn we denote the set of all permutations of [n]. For -ir £ Sn and x G {0, 1}" we abbreviate 
7r(a;) := x^(^i^ . . . x^(„). All logarithms log are binary logarithms. 

Definition 1 (fc-ary unbiased operator). Let A; G N. A k-ary unbiased distribution (D(. | 
y^^\ . . . ,y^^^)) {1) y(fe)g|oi}i ^-5 a family of probability distributions over {0,1}" such that for all 
inputs y^^', . . . , y^^' G {0, 1}"" the following two conditions hold. 

(i) Vx, z G {0, 1}" : 1>{x I y(i) , . . . , y^'''>) =V{xez\ y^^) © z, . . . , y^''^ © z) , 
(n) Vx G {0, ir Vvr G 5„ : P(x | y^), . . . ,y«) = V{7t{x) \ 7r(y«), . . . ,7r(2/W)) . 

We refer to the first condition as ©-invariance and we refer to the second as permutation invariance. 
A variation operator creating an offspring by sampling from a k-ary unbiased distribution is called 
a k-arj unbiased variation operator. 

Note that the combination of ©- and permutation invariance can be characterized as invariance 
under Hamming-automorphisms: P(- | ■,...,•) is unbiased if and only if, for all a : {0, 1}" — )■ {0, 1}" 



Algorithm 1: Scheme of a fc-ary unbiased black-box algorithm 



1 Initialization: Sample x^^' € {0,1}" uniformly at random and query f{x^^')] 

2 Optimization: for t = 1, 2, 3, . . . do 
Depending on (/(x*-")), . . . , f{x^^^^')) choose k indices ii, . . . ,ik € [0..t — 1] and a /c-ary 
unbiased distribution (P(. | y^^\ . . . ,2/^'^^))y(i)^...^y{fe)g{o,i}"; 
Sample x^^' according to P(. | x^"^^' , . . . jX^**^) and query f{x^^'); 



preserving the Hamming distance (i.e., \{j € [n] \ Xj = yj}\ = \{j £ [n] \ {a{x))j = {a{y))j}\ for 
all x,y) and all bit strings y^^' , . . . ,y^^\x € {0,1}" we have 'D{x \ y^^\. . . ,y^^') = T){a{x) \ 
a(y(i)),...,a(y«)). 

1-ary operators, also called unary operators, are often referred to as mutation operators. In 
fact, the standard bitwise mutation operator (as used, e.g., by many evolutionary algorithms) is a 
unary unbiased variation operator. 

2-ary operators, also called binary operators, are typically referred to as crossover operators. 
The classic uniform crossover operator is an unbiased binary one. Given two search points x and 
y, the uniform crossover operator creates an offspring z from x and y by choosing independently 
for each index i € [n] the entry Zi G {xj,yi} uniformly at random. However, the standard one- 
point crossover operator — which, given two search points x,y € {0, 1}" picks uniformly at random 
an index k € [n] and creates from x and y the two offsprings x' := xi . . . x^yk+i ■ ■ -Un and y' := 
yi . . . yk^k+i ■ ■ ■ Xn — is not permutation-invariant, and hence not an unbiased operator. 

A /c-ary unbiased black-box algorithm is an algorithm following the scheme of Algorithm 1 . 

2.1 Unbiased Black-Box Complexity 

Note that Algorithm 1 runs forever. This is justified by the fact that as performance measure 
of a black-box algorithm we take the number of queries performed by the algorithm until it first 
queries an optimal solution ("first hitting time"). More precisely, we consider the expected number 
of such queries as we regard random algorithms. This is the standard performance measure for 
randomized search heuristics, because in typical applications of such heuristics, evaluating the 
fitness of the search points is more costly than the generation of new ones. 

Formally, for a /c-ary unbiased black-box algorithm A and a function / : {0, 1}" — )■ M, let 
T{A, /) € M U {oo} be the expected number of fitness evaluations until A queries for the first time 
some X € argmax/. We call T{A, f) the runtime of A for f. 

Following the usual approach in complexity theory, for a class J- of functions {0, 1}" —^ M, the 
A-black-box complexity of T is r(A, J^) := supjgjrT(^, /), the worst-case runtime of A on T. 

The k-ary unbiased black-box complexity of J- is T{A,J-) := inf>ig^T(A, J^), the minimum 
( "best" ) complexity among all ^4 € ^ for J^, where we denote by A the class of all k-ary unbiased 
black-box algorithms for the functions in J^. That is, the /c-ary unbiased black-box complexity 
of some class of functions J- is the complexity of J-" with respect to all /c-ary unbiased black-box 
algorithms. 



3 The Black-Box Complexity of OneMax 

One test function often regarded in the randomized search heuristics community is the so-called 
"OneMax" function Om, which simply counts the number of 1-bits, Om(x) = X^ILi-^*- ^^ ^^ ^ 
linear pseudo-Boolean function with all bit weights set to one. Therefore, we consider Om one of 
the simplest linear pseudo-Boolean functions. 

Much theoretical work has been done for this function. In fact, already one of the first theoretical 
works on the (1 + 1) evolutionary algorithm (EA) [Miih92] studies the runtime of this algorithm 
on Om. In [DJWIO] it has been shown that, in fact, Om is not only considered one of the simplest 
linear pseudo-Boolean functions but that the runtime of the (1+1) EA on Om is at most as large 
as it is on any pseudo-Boolean function with a unique global optimum. 

The natural generalization of Om to a non-trivial class of functions is as follows. 

Definition 2 (OneMax„). Let n G N. For z e {0, 1}" let 

Om, : {0, 1}" ^ [0..n],x ^ \{j G [n] \ Xj = zj}\ , 

the function that counts the number of positions in which x and z agree. The string z = arg max Om, 
is called the target string o/Om,. 

Let OneMax„, := {Om, | z G {0, 1}"} be the set of all generalized OneMax functions. 

The OneMax„, function class has been studied in several contexts. For example, Erdos and 
Renyi studied it already in the sixties in the context of coin- weighing problems [ER63] . They show 
that t = (1 + o(l))(2n/logn) queries to Om, are necessary and sufficient to determine a fixed 
(unknown!) target string z, with high probability. 

The results of Erdos and Renyi were rediscovered by Chvatal [Chv83] in his studies on the 
Mastermind problem. Chvatal generalizes the upper bound given in [ER63] and shows that for any 
constant number A; G N, 0(n/logn) queries are sufficient to identify any /c-color string z G [k]"'. 
In his Mastermind model, queries are A;-color strings x G [fc]" and the functions that needs to be 
optimized are 0M2 : [k]^ — ;• [0..n],a; i-^ \{j G [n] \ Xj = Zj}\, z G [/c]". 

Doerr and Winzen [DW12b] show that the memory-1 restricted black-box complexity 
of OneMax„ is @{n/logn), disproving an earlier conjecture by Droste, Jansen, and We- 
gener [DJW06]. 

As mentioned, the results in [ER63] and [Chv83] seem to have been overlooked in the randomized 
search heuristics community for several years. For this reason, the lower bound of ^(n/logn) was 
rediscovered by Droste, Jansen, and Wegener in [DJW06, Theorem 4] and the upper bound of 
(1 + o(l))2?i/logn was also proven by Anil and Wiegand in [AW09]. 

For the unary unbiased black-box model Lehre and Witt [LWIO] show that the complexity of 
OneMax^ is O(nlogn). For arities 2 < k < n the best known bounds are 0(n/ log A;) proven 
in [DJK+11]. We improve these bounds in the following theorem, which is the main result of this 
paper. 

Theorem 3. Let n G N and let 1 < k < logn. The k-ary unbiased black-box complexity of 
OneMax„ is 0{n/k). 

For constant values A;, the result from [DJK^ll] shows that the A;-ary unbiased black-box 
complexity of OneMax„ is at most linear in n. Therefore, we need to show Theorem 3 for non- 
constant k, i.e., for k = oj{l). 



3.1 Random Sampling 

The basis for most results on the black-box complexity of OneMax^ is the random sampling 
technique, which was discovered in the already mentioned work by Erdos and Renyi [ER63]. They 
show that if we take t = Q(n/logn) queries x^, . . . ,x^ from {0, l}'^ independently and uniformly 
at random, then, with high probability, only one possible target string remains. That is, they show 
that the set of all feasible target strings 

5fcas(^) := Si,,,{z,x\ ...,x'):={yG {0, 1}" | Vi € [t] : Om,(x*) = OMy{x')} 

has size 1, with high probability. 

Algorithm 2: The random sampling technique for maximizing OneMax„. 

1 Initialization: t^[(l + iigfl^)^ 

2 repeat 

3 

4 



for i = 1, . . . ,t do 

Choose x* from {0, 1}"" uniformly at random and query OMzix^); 

Set y -^ chooseConsisteiit(2;^, . . . , x*); 
Query OM^(y); 



7 until OMz{y) = n\ 



In [DJK^ll] it has been shown that this random sampling technique, for which we give its 
pseudo-code in Algorithm 2, can also be done unbiasedly. To this end, one has to show that the 
operator chooseConsistent(-, ...,•), which given t bit strings x^, . . . ,x* E {0, 1}"" samples from 
^feasl-z, 3;^, . . . , X*) Uniformly at random, is an unbiased one. 

The random sampling technique can be formalized as follows. 

Theorem 4 (Random Sampling Technique). Let n G N and let z € {0,1}". Set t := [(l + 
— °^ rf" )lcr^1 ~ (1 -|- o(l))2n/log77,. Ifx^,...,x^ are sampled from {0,1}" independently and 
uniformly at random, then with probability at least 1 — o(l) the set Sj^asiz, x^, . . . , x^) contains only 
the target string z itself. That is, for all y € {0, l}"\{z} there exists an index i € \t] such that 
Om^(x*) 7^ OMy(a;*). 

3.2 Derandomized Random Sampling 

For the improved upper bound on the A;-ary unbiased black-box complexity of OneMax„, we will 
need a derandomized version of the random sampling technique. This is what we describe next. 

Definition 5. Let n,t G N. A sequence x^,...,x* G {0,1}" is string-distinguishing if for any 
two length-n bit strings y,z £ {0,1}" with y ^ z there exists an index i = i{y,z) G [t] such that 
Om^(x*) 7^ OMy(x*). 

Put differently, the sequence x^, . . . ,x* G {0, 1}" is string-distinguishing if for all strings z G 
{0, 1}" the set 5feas(-2) contains only z. 

The problem of finding string-distinguishing sequences was also considered in the work by Erdos 
and Renyi [ER63]. In fact, they prove the following statement. 



Theorem 6 (Derandomized Random Sampling [ER63]). For any 5 > there exists a positive 
integer no S N such that for all n > hq there exists a string- distinguishing sequence r^, . . . ,r^ G 
{0,1}" witht = (l + 5)log(9)n/log(n). 

The proof of Theorem 6 uses the probabilistic method and hence, is a non-constructive one. 
That is, although we know that such string-distinguishing sequences of length t exist, the proof 
by Erdos and Renyi does not show how to explicitely construct such a sequence. However, this 
does not matter to us: Since in black-box complexity we count only the number of queries needed 
by the "best" algorithm to optimize OneMax„, we may assume that our algorithms, given the 
problem size n, first do some preprocessing to compute a string-distinguishing sequence r^, . . . ,r*. 
This can be done deterministically. The preprocessing itself does not require any query, because 
the algorithm can check without querying anything, whether for any two bit strings y,z ^ {0, 1}" 
with y ^ z there is at least one index i G [t] such that OM2(r*) ^ OMj^(r'). 

To conclude this section, we briefly note that log(9) is roughly 3.17. Since we are not in- 
terested in constant factors here in this work, we settle for the fact that, for large enough n, 
string-distinguishing sequences of length t = 3.5?i/logn exist. 

3.3 Invariance of String-Distinguishing Sequences 

Since we are dealing in this paper with unbiased black-box complexities, we need to ensure that 
we employ only unbiased variation operators. In the unbiased model we can (typically) not ac- 
cess particular bit positions nor can we require a particular bit value to be set to 1 or to be set 
0, respectively. However, we can work around this technical difficulty. One ingredient of this 
work-around is the observation that the set of string-distinguishing sequences is invariant under 
Hamming-automorphisms. 

Theorem 7 (Invariance Property). Let n £ N be large, let t = 3.5n/logn, and let r^,...,r* be 
a string- distinguishing sequence. For all w G {0, l}" and all permutations vr of [n] the sequence 
7r(r^ © w), . . . , 7r(r* © w) is string- distinguishing. 

Proof. Let w G {0,1}" let vr be a permutations of [n]. Let y,z G {0,1}". Since r^,...,r^ is 
string-distinguishing, there exists an index i G [t] such that 

OM^-l(^)gj^(r*) / OM^-l(j^)g5^(r*) . 

We show that 

OMz{Tr{r'- © w)) ^ OM.y(7r(r' © w)) . 

Observe that for all j G [n] we have (7r~^(z) © w)j = r*- if and only if z^-i/j\ = {7r~^(z))j = 
(r* © w)j = (7r(r* © w))^-i(-j). From this we conclude 

OM^(7r(r* ©u;)) = 0M^-i(^)g5^(r*) 

/ OM^-i(,y)ge^(r^) = OMj,(7r(r^ © w)) . 



4 The Encoding Technique 

The main contribution of this work is an encoding technique which we present in this section. We 
show that using only k-ary variation operators, we can create a storage of size 0,(2^). This storage 
can be used to encode information. Here in our apphcation of maximizing OneMax„ functions we 
shall use this storage to encode fitness values OM^(a;). As was done in [DJK"*"!!], we identify z by 
repeatedly identifying parts of z. This is the block- wise optimization technique which we briefly 
summarize below. The details of the encoding technique are presented in Sections 4.2 and 4.3. 

4.1 Block- Wise Optimization 

The paper [DJK+ll] introduced a block- wise optimization technique for OneMax„. We say that we 
"optimize" a block of length i if we identify the entries Zjj , . . . , Zi^ for some subset {ii, . . . ,ie} ^ [n] 
of size i. The main idea of the block-wise optimization technique is to split the full length-n bit 
string into blocks of length i and to sequentially optimize these blocks. Each of these length-^ blocks 
is optimized in 0(^/log^) queries via the random sampling technique that we have presented in 
the previous section. Since there are [n/£] such length-^ blocks, the total number of queries needed 
by this algorithm is 0(n/log^). 

Here in this paper we shall also apply block-wise optimization. For fixed arity k, we set k := k—7 
and i := 2^. We show how to optimize blocks of length i. Before we present the details, let us 
note already here that while i is of order Q{k) in [DJK"*"!!] (thus yielding an 0{n/logk) upper 
bound for the k-ary unbiased black-box complexity of OneMax„), our choice oi i = 0(2'^) yields 
the claimed 0{n/k) upper bound. 

Assume for the moment that we have created two bit strings x,y £ {0, 1}" such that for all 
bit positions in A{x,y) := {j G [n] \ Xj = yj}, in which x and y agree, we know that Xj = Zj. 
We call the positions in A{x,y) "optimized". The idea is that we need to care only about the 
positions D{x,y) := [n]\A{x,y), in which the two strings disagree. The strings x and y are a very 
condense way of encoding the information which bit positions are optimized and which are not. 
This encoding has been introduced in [DJK"*"!!]. We show how to create such strings later, but — as 
will be discussed below — in the beginning we may simply set y := x. 

In the following we first describe how we create from x and y a storage of size Ai = 2'^+^. We 
then show how this storage can be used for optimizing a block of length i. As mentioned, this is 
the core idea in the proof of Theorem 3 and we refer to it as the encoding technique. 

4.2 Creating the Storage 

We describe an unbiased algorithm that, given x and y, creates k + 3 bit strings y*^, . . . , y^"*"^ such 
that we can access 4i bits of x by an unbiased operator. 

First we need to identify the 41 bits which we want to use for storing. We define a binary 
operator f indStorage(-, •), which, given x and y, creates from x a string y^ such that D{x,y^) has 
size Ai. That is, our storage space of size Ai will be encoded in D{x,y^). We require that at least 
min{£, \D{x,y)\} bits of the storage are taken from the set D{x,y) of not yet optimized bits. To 
this end, we first choose uniformly at random from D{x,y) a subset B' of size min{£, \D{x,y)\}. 
Think of B' as the length-^ block we aim at optimizing. Next we choose uniformly at random from 
[n]\i?' a subset S of size Ai — \B'\. For the moment, we are not interested in optimizing the bits in 



S but rather need these bits for storing information. We set 

y° ^ findStorage(x,y) := xe ^ e" , 

jeB'us 

that is we create y^ from x and y by flipping in x the positions in B' and 5. The probabihty 
distribution defined by f indStorage(-, •) is an unbiased distribution. We omit a detailed verification 
of the unbiasedness of this distribution. The interested reader can find the unbiasedness proofs for 
all operators used in this work in the appendix. 

By construction we have D{x, y^) = B' U S, and B' C D(x, y^) n D{x, y). Therefore, the size of 
D{x,y^) n D{x,y) is at least min{£, \D{x,y)\}. We shall need this observation later. 

In this section we only allow to flip bits in D := D{x,y^). Therefore, we set 

V := V{x, yO) := {w € {0, 1}" \yj ^ D : Wj = x^} , 

the set of possibly to be queried bit strings. 
For all u; G "P we define 

F''^\x,y°,w) := {j e D \ Wj ^ Xj} and 
f(°)(x,2/°,'u;) :=D\F^^\x,y°,w). 

Furthermore, we set 

Z(x,y,y°) ■= {w eV\ |F(i)(x,y°,w;)| = 2i,\F^°\x,y° ,w) n D{x,y)\ > mm{i,\D{x,y)\}} . 

The set Z{x,y,y^) contains all the strings in V which can be created from x by flipping exactly 2i 
bits, keeping at least min{^, \D{x,y)\} bits in D{x,y) constant. 

We sample y^ € Z{x, y, y^) uniformly at random. This can be realized via a 3-ary unbiased 
operator (details are given in the appendix). 

Assume that we have sampled y^, . . . ,y^ G "P for some s G N. We describe how to sample y*"*"^. 
For all w ^ V and for all vectors i G {0,1}* we set 

F^^''\x,y°, ...,y\w) := {j G r(x,y^, . . . ,y*) | ^,- ^ y]} , 
F(*'0)(x,yO, . . . ,y^u;) := {j G F^(x,yO, ...,yn\wj = y]} , 
Z(x,y,yO,...,y*):={u;GP|V.G{0,ir :|F(*'^)(x,y^...,y^u;)| = |F*(x,y^...,y*)|/2}. 

That is, w G Z(x, y, y^, . . . , y^) if and only if for all i G {0, 1}'' the two strings w and y'^ disagree 
in exactly half of the bits in F*(x,y'^, . . . ,y**). Only for the case s = 1 we need an additional 
requirement and we set 

Z{x,y,y^,y^) := {w G P | Vi G {0,1} :\F^''^\x,y^,y^,w)\ = ^ and 

\F^^''^\x,y'',y\w)nD{x,y)\>mm{e,\D{x,y)\}}. 

Note that by the observation made above we have \D n D(x,y)\ > min{^, \D{x,y)\} and, hence, 
the set Z{x,y,y^,y^) is non-empty. The same is true for Z{x,y,y^, . . . ,y^) with 2 < s < k + 2. 
We create y*"*"^ by sampling uniformly at random from Z[x,y,y^, . . . ,y^). Similarly as above this 
is a 4-ary unbiased variation operator for s = 2 and, as was proven in [DWll, Section 5], it is 



an {s + 2)-ary unbiased variation operator for s > 3. As mentioned above, details regarding the 
unbiasedness can be found in the appendix. 

We stop this process once it holds for all i G {0,1}'^ that the set F^{x,y'^, . . . ,y^) contains 
exactly one element. Since the size of D is Ai = 2'*"''^, this is the case when s = k + 2. 

In what follows, for all < s < k + 2 and all i € {0, 1}'^ we abbreviate F"^ := F^{x, y^, . . . , y^) . 

4.3 Accessing the Storage 

We now describe how to access the storage space D = B' U S oi size 4£. We first introduce a 
bijection between the length-(K + 2) bit strings and the positions in D. To this end, we recall the 
definition of the so-called "binary-value" function 

K + 2 

Bv : {0, l}'^+2 -^ [O.Ae -l],iu^Y, '^^^^'^Wj . 

j=i 

It is well-known that Bv is a bijection. 

Since for all i € {0, l}'^+2 the set F* has size one and F* ^ F^ for all i ^ j, the function 
a : [4£] — >• D, which maps each value i G [Ai] to the element in the set F^ (*-i))^ defines a 
bijection. Formally, a := a{x,y,y^, . . . ,y''"^^) depends on the random experiment described above 
but we omit this explicit notation for reasons of space. 

Note that, by construction, we have a{i) G D n D{x,y) for all elements i G [min{^, |L'(a;,y)|}]. 
This is exactly the reason why we had added the requirements \F^^'{x,y^ ,w) n D{x,y)\ > 
min{^, \D{x,y)\} and \F^^'^> {x^y^ ,y^ ,w)r\D{x,y)\ > min{^, \D{x,y)\} in the definitionof Z(x,?/,y'^) 
and Z{x,y,y^,y^), respectively. The block B := {cr(l), . . . ,cj(min{£, |D(x,y)|})} is the one we aim 
at optimizing. Note that B = B' may or may not hold. Note further that B = F^^'^>{x,y^ ,y^ ^y"^). 

We can now describe how to access the storage D. 

For all length-n bit strings x, y,y^, . . . , ^"+2 let £^{x, y,y^, . . . , y'^^'^) be the event that all of the 
following statements are correct: 

. \D[x,y^)\=U, 

• \D{x,y^) r}D{x,y)\ > min{£, \D{x,y)\}, and 

. ViG[K + 2]:r GZ(x,y,yO,...,y^-i). 

Abusing notation, for any such sequence of bit strings let a := cr(x, y, y*^, . . . , y'^^'^) be the bijection 
that maps each i G [M] to the (unique) element in F^^^ ^"^"^''{x, y^, . . . , y^^^). 

Let further s G [4£], r G {0, l}^ P C [A£] with |P| = s and tt; G {0, 1}". For ah j G [n] with 
d'~^{j) G P let i^d'~^{j) G [s] be the rank of element (T~^(j) within the increasingly ordered set P, 
i.e., 

i^a-Hj) := 1 + \{p e P \ p < d-Hj)}\ ■ 

Let <f ^(r, P,w,x, y, y^, . . . , y'^~^'^){w) be the event that all of the following statements hold 

• event £'^(x,y,y'^, ... ,y''"'"^) holds, 

• for all j G [n] with ^"^{j) G P and r^^-iij\ = as well as for all j G {n\ with d~^{j) ^ P we 
have Wj = Wj, 
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Wi. 



• for all j € [n] with a (j) € P and r^t^- 1(,) = 1 we have ■Wj = 1 
We set V{- I r,P,w,x,y,y'^,...,y''+'^) : {0,1}" ^ [0,1], tt; h^ 

2~" , if event £^{x, y, y°, . . . , y"'^^) does not hold, 

1 , if event £'^(r, P, w, x, y,y^, . . . , y'^~^'^){w) does hold, 

, otherwise. 

For proving that for all P C [M] and all r € {0, 1}' ' the family 

[vi-\r,P,w,x,y,f,...,r'-^)' 



■w,x,y,y°,...,y'^+^£{0,l}" 

is an unbiased distribution, observe first that, if event £^{x,y,y^, . . . ,y'^~^'^) holds, then the event 
£'^(r,P,w,x,y,y^, . . . ,y'^^'^){w) holds for exactly one bit string w. This shows that the function 
P(- I r, P, w, X, y, y^, . . . , y'^^'^) defines a probability distribution on {0, 1}". 

To show the unbiasedness of this distribution, we need to show that for all permutations vr G S^ 
and all strings v,w & {0, 1}" that 

• £^{x,y,y^,... ,y'^^'^) holds if and only if 6'^(7r(x v),7r{y f),7r(y° f),... ,7r(y''+^ v)) 
holds, and that 

• £'^{r,P,w,x,y,y^, . . . ,y^+^)(u;) holds if and only ii £'^{r,P,Tr{w®v),Tr{x®v),Tr{y<3)v),TT{y^ ® 
v),... , 7r(y'"+2 v)){7r{w v)) holds. 

This computation is straightforward. It follows essentially from the two facts that (i) for all 
permutations n G Sn and all strings x,y,v S {0, 1}" we have 

D{tt{x v), TT{y v)) = {^(j) I j G D{x, y)} 

and that (ii) 

a(7r(5 u), 7r(j/ © v), 7r(j/° u), . . . , vr(y'^+2 ?;)) = tt o a{x, y, y°, . . . , y'"+2) . 

We omit the details. 

The operator (write(r, P)) {x,y,y^, . . . , y'^~^'^,w) is the one that samples from this (k + 6)-ary 
distribution. 

Let us briefly conclude that here in our context, for any set P C [4^], any string r € {0, 1}'^', 
and any w G {0,1}", the operator {yir±te{r,P)){x,y,y^,...,y'^~^'^,w) is deterministic. That 
is, given r,P,w,x,y,y^,...,y'^~^'^ it always outputs the same bit string, which we abbreviate by 
write(r, P, If). The operator does so by setting, for each j G [n] the value (write(r, P, u;)) . to 

Wj , if a-^{j) G P and r^a-^j) = , 

1 - WaU) ' if <7-^(j) G P and r#a-m) = 1 > 
u;,- ,ifa-\j)^P. 
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5 Proof of the Main Theorem 

Throughout this proof, let n G N be large and let k € [logn] na;(l). Let I := 2'^ for k := k — 7. Let 
further t = 3.51/ log£. By Theorem 6 there exists a string-distinguishing sequence r^, . . . ,r* for bit 
strings of length i. 

In what follows, instead of writing min{£, \D{x,y)\}, for convenience, we simply write i. 

5.1 Computing the Contribution of the r"s 

As mentioned, the algorithm verifying Theorem 3 works by repeatedly optimizing blocks of length 
i. Let us assume that we are currently interested in optimizing block B, that is, we are interested 
in identifying the entries of z in the positions B. We would like to do so by querying r^, . . . ,r*. 
However, this bears two difficulties: the length of each r* is only i instead of n and, secondly, we 
are only allowed to query bit strings that can be created by using unbiased variation operators. In 
particular, we cannot request a particular bit value to be set to or to 1, respectively. 

We show how to work around these obstacles. To this end, we first query a reference string 
encoding the current block we are interested in. This is the string 

y^ ■■=x(B^e'l. 

i€B 

Creating y^ can be done via the 4-ary unbiased operator which, given x, y^, y^,y^ ffips in x all bits 
in F(°'°)(a;,y°,y\y^) and sets yf = Xj for ah j ^ F'^^'^\x,y'^ ,y^ ,y'^). 

Note that we can compute the contribution of the bits outside of B from the fitness values of x 
and y^ by the equality 

\{j G [n]\B I X, = Zj}\ = (OM,(y^) + Om,(x) - £)/2 . 

This equation holds because the bits in B contribute exactly I to the sum Om z{y) + Om^(2;) and 
all other bits contribute either or 2. 

After querying y^ we query the fitness values of write(r-'^, [^],x), . . . ,write(r*, [^],x). By the 
observation made above, we can infer 

Al,:=AB(write(r\[^],x)) 

■.= \{j&B\{^,r±te{r\[l],x))^=Zj}\ 

= OM,(write(r^ [£],x)) - (OM,(y^) + Om,(x) - l)/2 

from the fitness values OM2(write(r*, [i]^x)),OMz{y^)^ and 0m2(x). 

Clearly, A^ G [0..^]. Thus, k + I bits are sufficient to store each such value in binary. Note that 
we sample t = 3.5£/log£ = 3.5^/k strings write(r*, [P\,x) in total. Thus, a total storage of size at 
most 3.5£(k + 1)/k < Ai = \D\ is sufficient to store all values A^, . . . , A^. 

In what follows, for any r G [0..£] we write Bv~_|_-^(r) for the string that is created from Bv~ (r) 
by removing the first entry (which equals 0). This is a length-(K + 1) bit string. 

For storing the values A^, . . . , A^ we initialize s ■(— x and then repeatedly update s by applying 
the operator write. More precisely, for i running from 1 to i we update s by 

s ^ write(Bv;^i(A*B), {(i - 1)(k + 1) + 1, . . . , i{K + 1)}, s) . 
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Algorithm 3: Optimizing OneMax„ with unbiased variation operators. 

1 Initialization: Sample x € {0, 1}" uniformly at random and query Om2(x); 

2 Set y -^ X and query OM^(y); 

3 for block 6 = 1,..., [n/£] do 

4 
5 
6 

7 



8 

9 

10 

11 

12 
13 
14 
15 
16 



Create a storage D of size M (cf . Section 4) ; 
Query OM^(y^); 
for i = 1, . . . ,t do 
I Query OM2(write(r*, [£],rE)); // query string-distinguishing sequence 

Set s <— x; 

for i = 1, . . . ,t do 

Update s ^ write(Bv;:]:^(A*^), {(i - 1)(k + 1) + 1, . . . ,i{K + l)},s); 

Query OMz{s); / / store fitness values 

repeat 

Sample q ^ chooseConsistent(x,2/^, s); 

Query OM^(g); 
until Asiq) = i; 
Update X and y; 



The resulting string s encodes all values A^, . . . , A^ and, as we shall see below, it allows us to regain 
the full guessing history. Note that the operator write(Bv~^;^(A^), {(i — 1)(k-|-1) + 1, . . . ,i{K+l)}) 
can be chosen by knowing only the fitness values OMz{x),OMz{y^), and OM2(write(r*, [£],a;)). It 
is important to note that the strings y and write(r*, [f],x) do not count toward the arity of the 
operator creating 

write(Bv;;i(A*B), {{i - 1){k + 1) + 1, . . . , i(K + 1)}, s) 
= (write(Bv;^i(A^), {{i - 1){k + 1) + I, . . . ,i{K + l)}))(x,y,yO, . . .,y^+^s) . 

5.2 Optimizing OneMcLK„ 

Algorithm 3 gives an overview of the algorithm verifying Theorem 3. 

In this algorithm we make use of the operator chooseConsistent(x,y , s) which we describe 
next. Given the strings x, y, y*^, . . . , y"'^^, y^, and s we first compute 

J"b := {w G {0, 1}" |Vj G A{x,y^) : Wj = Xj and 

Vie [t] : OMz{iir±te{r\[i],x)) = OM^„(write(r*, [£], x)) } . 

The operator chooseConsistent(a;,y , s) samples from this set uniformly at random if it is non- 
empty and it outputs a random string otherwise. This is an unbiased operator. We refer to 
the appendix for a proof of the unbiasedness and note here only that the arity of the operator 
chooseConsistent is k -|- 7. 

To compute J-b, the operator chooseConsistent needs to infer the fitness values 
OM2(write(r*, [£], a;)). We show that this is possible by accessing only the information given 
in the strings x, y, y^, . . . , y'^~^'^,y , s and their corresponding fitness values: For any two numbers 
o, 6 € M define the Kronecker symbol by setting 5{a,b) = 1, if a = 6, and 6{a,b) = 0, otherwise. 
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The operator chooseConsistent can infer the value OM2('write(r*, [^],a;)) from the input strings 
by first computing 

K+l 

^B = ^2''+^~^(l - (5(x^((j_i)(«+i)+j),s<^((i„i)(K+i)+j))) 
i=i 

and then adding to A^ the value {OMz{y ) + Om2(2;) — i)/2. This shows how to compute J-b- 

Note that here in our case we actually have \Tb\ = 1- This is by definition of the sequence 
r^, . . . , r* and Theorem 7. In particular we have (chooseConsistent(x, y , s))j = Zj for all j € B, 
and, again by definition, we have (chooseConsistent(x, y^ , s))j = Xj for all j G [jt^JV^ = A{x, y^). 
As mentioned in the beginning of Section 4, in Line 16 of Algorithm 3 we want to update x and 
y in such a way that we preserve the invariance that Xj = yj only for such j for which we know 
that Xj = Zj. To maintain this invariance, we first update y by setting for each j G [n] 



Vj , if yf = Xj , 

chooseConsistent(a;, y , s)j , ii yj ^ Xj . 



Confer the appendix for a (straightforward) proof that y can be updated by an unbiased variation 
operator. The arity of this update operator is 4 as it requires x, y, y^ , chooseConsistent(a;, y^ , s) 
as inputs. 

After updating y, we update x by just copying 



x ^ chooseConsistent(a;,y , s) . 

5.3 Runtime of Algorithm 3 

It remains to bound the runtime of Algorithm 3 by 0{n/k). To this end, we first observe that in 
Algorithm 3 the operator of largest arity (this is the chooseConsistent operator) has arity k + 7. 
This shows that all variation operator are of arity at most k. 

Next we observe that the (deterministic) number N of queries that Algorithm 3 
does to optimize a block of length £ (Lines 4-16) is k + 2t + 6 (the queries are 



y 



,,K+2 



,2/'^, write(r-^, [£],x), . . . , write(r*, [^], j;) plus t queries for updating s, the query 
chooseConsistent and one query for updating y). Now t = 3.5£/log£ < 2'^~*"^/k; = 2^~^ /{k — 7) 
and, hence, N < 2^-'^/{k -7) + k-l. 

There are \'n/€\ = \n/2^~'^^^ blocks of length (. and we have two additional queries in the 
beginning for initializing x and y. Thus, Algorithm 3 does a total number of less than 

^-^ ' +A;-l)+2 



"/^ v.- 7 

queries. This term is in 0{n/k) for k G [logn] na;(l). 

6 Conclusion 

We have shown that the /c-ary unbiased black-box complexity of OneMax„ is at most 0{n/k) for 
1 < A; < log n, thus greatly improving the previously best known bound of 0{n/ log k). This results 
motivates further research in three directions. 
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An obvious question arising from our result are matching lower bounds. So far, the only lower 
bounds known are ri(n log n) for unary unbiased black-box algorithms and il.{n/logn) for the 
unrestricted case. Hence we currently cannot rule out that already the binary unbiased black-box 
complexity of OneMax„ is 0(n/ log n). It seems that proving lower bounds stronger than those 
for the unrestricted setting and arities greater than one need the development of substantially new 
methods. 

Both the previous 0{n/logk) and in a stronger sense our new 0{n/k) upper bounds for the 
/c-ary unbiased black-box complexity can be seen as an indication that using higher arity variation 
operators may increase the performance of randomized search heuristics. This seems to be a 
research problem for which still few theoretical results exist. There is a sequence of papers showing 
that a particular evolutionary algorithm for the all-pairs shortest path problem becomes more 
efficient when a natural crossover operator is used [DHK08,DT09,DJK"^10]. There also is a recent 
result [KSTll] on OneMax, which, however, uses an uncommon shuffling operator that clearly 
is far from unbiased. The same paper, as well as classic works by Jansen and Wegener [JW02] 
show that crossover can be useful for jump functions, if population size, jump parameter and 
crossover rate are in a favorable relation. Despite these results, it seems that even for simple test 
functions like OneMax we do not understand crossover very well, and nothing seems to be known 
on non-artificial uses of higher arity operators. 

Finally, the key methods to derive the superior black-box optimization algorithm presented 
in this work is a quite technical encoding technique that allows to simulate a storage of 2'^ bits 
in the unbiased black-box model. While we are optimistic that this technique will lead to some 
more unexpected black-box results, this elaborate technique also raises the question to what extent 
unbiased black-box complexities are a suitable measure for how difficult a problem is for a reasonable 
search heuristic. While we do feel that the different black-box models suggested in the recent past 
greatly enlarge our understanding of problem difficulty, it remains a challenging problem to find 
even better complexity notions. 
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A Unbiasedness of Variation Operators 

We show that the variation operators used by Algorithm 3 are unbiased. We do that in the sequence 
they appear in the pseudo-code of Algorithm 3. 

A.l Sampling x 

In the first line of Algorithm 3 we sample x 6 {0,1}"'. This is an unbiased (0-ary) operator. In 
fact, it is the only 0-ary unbiased variation operator on {0, 1}". 

A. 2 Sampling y 

In the second line of Algorithm 3 we set y to be the bit-wise complement of x. This is a unary 
unbiased variation operator as can be seen as follows. For all x,w £ {0, 1}" it holds that x (B w = 
X (B w and for all x and for all vr S S'n we have 'jr{x) = 7r(x). 
The corresponding distribution is 

„/ , s r ^« r T I 1 , ii yj £ \n\ : Wj = 1 — Xj , 

Vi.\x):{0,ir^[0,l],w^\ J } ^ ' ^' 

I U , otherwise. 

A. 3 Sampling y^ 

We show that f indStorage(-, •) is an unbiased variation operator. To this end, we define for all 
bit strings x,y £ {0, 1}" the distribution 

~, , ■. r ^« r -, |0 ,ifw4S(x,y), 

Vi-\i,y):{0,ir^[0,l],w^\ ' ^ ^ J ^, ' 

[\'S{x,y)\ ,itwe S{x,y), 

where we abbreviate 

S{S:,y) :={w G {0,1}" | \D{3:,w)\ = Al] and \D{i,y) r\D{i,w)\ > min{^, |L»(x,y)|} . 

Clearly we have for all permutations vr of [n] and all z € {0, 1}" 

D(7r(x e z), 7r(y © z)) = {^{j) \ j € D{x, y)} (1) 

and thus, \D{x,y)\ = |D(7r(x © z),7r(y © z))|. 

Therefore, it is easily verified that for all w € {0, 1}" we have w € S{x, y) if and only if 
■k{w © z) G S[tt{x © z),Tr{y © z)) . Hence, 

V{w \ x,y) = 'D{t:{w © z) \ t^{x © z),TT{y © z)) . 

This shows that (P(- | x,y))^y^^Qijn is an unbiased distribution. 
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A. 4 Sampling y^, . . . , y'^^'^ 

We show that sampling y^ is a 3-ary unbiased operation. The unbiasedness of saniphng y^, . . . , y''+2 
can be proved in a similar way. 
Recall that we have set 

Z{i,y,y^) = {weV\\F^^\x,y'',w)\ = 2^F^''\x,y'',w)r^D{i,y)\>mm{^D{xM}}■ 
For all X, y, y^ S {0, 1}" we define 

(2- ,ifyV5(x,y), 

V{- I X, y, f) : {0, 1}" ^ [0, 1], u; ^ <^ , if f G 5(x, y) and w ^ Z{x, y, f) , 

[\Z{x,y,y°)\~^ , if y° e S{x,y) and w e Z{x,y,y^) . 

Using again equation (1) one easily shows that 

w G Z{x,y,y^) 

if and only if 

Ti{w © z) G Z(7r(x © z), 7r(y © z), 7r(y° © z)) 

for all permutations vr G S'n and all z G {0, 1}". 

Together with the already mentioned fact that y" G S{x, y) if and only if TT{y^ © z) G S{t:{x © 
z),TT{y © z)) this shows the unbiasedness of sampling y^. 

A. 5 Unbiasedness of Creating y^ 

Creating y^ is equivalent to sampling from the following distribution T){- \ ■,-,•,•) with input 

For all x,y^,y^,y'^ we let 



(112 

x,y",y ,y 



Vi- \i,f,y',y') : {0,ir ^[0,l],w ^ { 



'l , if Vj G F(0'0)(i, yO, yi, y2) .^.^i_ ^. 

and Vj G [n]\F^^'^>{x,y^ ,y^ ^tp') : Wj = Xj , 
, otherwise. 



Again it is straightforward to show that this is an unbiased distribution. It follows essentially 
from the fact that 

f(O'O) (7r(x © «;), 7r(y0 © w),7T{y^ © w),7T{y^ © w)) = {7r(j) | j G F^^^'^^x, y\y\f)] 

A. 6 Unbiasedness of the Operator write 

This operator has been described in the main text in much detail, cf. Section 4.3. 
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A. 7 Unbiasedness of chooseConsistent 

Let X, y, y^, . . . , y'^~^^, y^ , s be length-n bit strings. We say that event £^ holds for these strings if 
all of the following events are true: 

• Sj = Xj for all j G A{x,y^). 

Recall that for such strings we have defined 

:= {w G {0,1}" I Vj € A{i,y^) : Wj =ij,'ii G [t] : /(write(r*, [£],x)) = OM^{vr±te{r\[£],x))} 
where 

re+l 

/(write(r\ [i],x)) = ^ 2'*+^-^ (l - 5(x<j((i_i)(«,+i)+j) , s^((i_i)(^+i)+j))) 

+ (OM,(y^) + OM,(x)-^)/2 

(a is defined as in Section 4.3). 

Let Vi- \ x,y,f,. . . ,y''+^,y^ ,s) : {0,ir ^ [0,l],w ^ 

2~" , if event £^(x, y,y^, . . . , y'^^'^ , y^ , s) does not hold, 

l^^l^-"^ , if event £^{x, y,y^, . . . , y'^~^'^,y^ , s) holds and w G Tb 
, otherwise. 

The unbiasedness of this distribution follows essentially from the fact that we sample from J^b 
uniformly. First we observe that event £^{x,y,y^, . . . , y'^~^'^,y , s) holds if only if for all permuta- 
tions IT £ Sn and ah v G {0, 1}" the event £^ (7r(x) © v, 7r(y) © v, 7r(y°) ®v,... , 7r(y'^+2) © v, n^y^) © 
v,tt{s) © v) holds: 

• For the events f ^ we have argued for this already in Section 4.3. 

• F(°'°)(7r(x) ffii;,7r(y°) ©i;,7r(y^) ®v,TT{y'^) ® v) = 7r(F(°'°)(x, y°, y^ y^)). Hence, 

if and only if 7r(y^) © t; = 7r(x) © i; © EieF(o,o)(^(x)(BvMy'')®vMy')(SvMy^)®v) < > 

• Clearly, Sj = Xj for all j G A{x,y^) if and only if (vr(s) © v)j = {tt{x) © v)j for all j G 
A{7r{x) ©f,7r(y°) ©v). 
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Therefore, we need to show that 

w £ Pb{x, y,f,..., y''+^ y^, s) if and only if (2) 

tt{w) ®v £ f {tt{x) e V, 7r(y) v, T:{jf) -y, . . . , 7r(2/''+2) © v, 7r(y^) v, tt{s) © v) . 

This fohows from the following two observations. For all u,v,w G {0, 1}" and all permutations 
vr G 5„ we have 

OM^(n) = OM^(^)g5^(7r(n) v) . 

Further we have 

Using these two facts one easily verifies (2). 

This shows that the distribution D(- | x,y,y^, . . . ,y'^~^'^,y^ ,s) is unbiased. The operator 
chooseConsistent(i,y , s) = chooseConsisteTi.t{x,y,y^, . . . ,y'^~^'^,y ,s) is the operator sam- 
pling from this distribution. 

A. 8 Unbiasedness of Updating y 

For all x,y,y^,w let 



V{- \ 3:,y,y'',w) : {0,ir ^ [0,l],w ^ { 



1 , if Vj € D{x, y^) : Wj = Wj 

and Vj € [n\\D{x, y^) : Wj = jjj . 
, otherwise. 



By equation (1) this is easily verified to be a 4-ary unbiased distribution. For updating y in 
Line 16 we sample from P(- | x,y,y , chooseConsistent(j;,2/ , s)). 
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